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OVERVIEW EVALUATION 
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"Evaluation is defined as the making of judgments about 
the value* for some purpose of edeas* works, solutions^ methods* 
materials, etc® St involves the use of criteria as well as 
standards for appraising the extent to which particulars are 
accurate, effective, economical or satisfying,, " The judgments 
may be either quantitative or qualitative, and the criteria may 
be either those determined by the student or those which are 
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given to him®" 

"A measure of educational achievement provides two kinds 

of informations The first is the degree to which the student 

has attained the educational objectives 0 . oThe second is the 

relative ordering of individuals with respect to their per- 

2 

formanceo fl 

The two previous items define evaluation and describe 
what is sometimes measured in such an evaluation,. Much has been 
written relating to the evaluation of the pupil, the programs, 
and the processes* Little has been written or done in the field 
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of vocational-technical education® Jerome Moss, Jr®, in a 
recently published document from the University of Minnesota, 
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September, 1963, comments concerning the importance of program 
evaluation in vocational-technical education* 

In light of its importance, how can the fact be 



explained that program evaluation in vocational, tech- 
nical, and practical arts education has been an inci- 
dental, casual, and sporadic activity? As a matter 
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of fact * why have relatively few evaluative studies 
been conducted since the passage of the Vocational 
Education Act of 1963? 

There are philosophical-political reasons® We 
are faced with critical manpower and social problems® 
The public policy decision has been to divert .lost 
available human and financial resources from both 
research and evaluation into the most visible 
approaches that can at least sustain the illusion 
of progress® With most public agencies under 
pressure to produce immediate results * it is no 
wonder that the need for a good show often over- 
whelms scientific objectivity; it is not surprising 
that there is little time to revise* throw out* and 
frankly compare® Careers are often at stake® Fur- 
ther, it is somehow un-American to be indefinite 
and doubtful* or to adopt a try-and-see attitude 
about any proposed public program® Legislators are 
loathe to provide large sums of money to try out 
several alternative solutions; we pick one "solu- 
tion" and go. Obviously* evaluation is done hesi- 
tantly* with very grave consequences usually asso- 
ciated with unfavorable findings® "Whan ideas that 
are promising as objects of research and honest ex- 
perimentation Lax*® accepted prematurely and] give 
birth, through artificial dissemincition* to a brood 
of hysterical fads* there is the danger that angry 
reaction will dump out the egg with the shell" 
(Oettinger* 1968* p. 76-77)® 

There are personnel reasons for the relative 
inactivity® Vocational* technical and practical 
arts education has been handicapped by a shortage 
of well-trained researchers* and evaluation has not 
been looked upon (mistakenly from my perspective) 
as a specially rewarding* creative form of research® 

Finally* there have been (and still are) techni- 
cal difficulties® The remainder of this paper will 
touch on some of these problems so they will not be 
enumerated here® Suffice it to say for the moment 
that evaluation is a highly complex* technically 
and conceptually demanding activity® Until relatively 
recently* we lacked the statistical and computational 
tools necessary to do a reasonable job® 

From now on* however* the picture must change® 

The Advisory Council on Vocational Education has 
demanded greater efforts at evaluation® Social 
scientists from a wide variety of disciplines are 
turning their attention to the assessment of various 
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systems of manpower training © Our social obli- 
gation for evaluation is being assumed by others* 
and the results could determine our very existences 
We must evaluate our own programs using appropriate 
criteria and methodology so that decisions concern- 
ing our future can be based upon data which properly 
reflects our educational perspectives© 



He further suggests that first, evaluation must be com- 
parative because evaluation involves making a judgment and 
second, diff erenr es in the outcomes compared must be attributed 
to program characteristics or the interaction of program and 
student characteristics© 

It becomes necessary, whenever the problem of evaluation 
is broached, to have on awareness of what it is that needs to 
be evaluated© Harold Starr 4 of Ohio State suggests the follow- 
ing as program goals that could be the subject of evaluation 
if it were desirous to evaluate a program of vocational- 
technical education© 

1» To provide vocational— technical education 
and training to youth and adults who will 
be entering the labor force and to those 
who seek to upgrade their occupational 
competencies or learn new skills© 

2© To provide comprehensive curricula which 
relate general and vocational— technical 
education offerings to the vocational 
objectives of students© 

3o To provide increased accessibility to pro- 
grams of vocational-technical education to 
meet the needs of those to be served© 

4. To provide quality instructional programs 
which meet the vocational aspirations of 
people while being compatible with employ- 
ment opportunities 
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5o To provide a systematic and continuous evaiua- j 

tion of vocational-technical education in J 

terms of national and state interests; student j 

benefit; and manpower requirements as a means j 

for making decisions concerning alternative \ 

investments of human and economic resources j 

and the redirection of program objectives o I 

I 

There seems to bs little question in the minds of con- j 

cerned vocational educators at local; state and national levels • 
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that evaluation of vocational-technical education is a "must” 

i 

\ 

because of the .level of federal involvement that is developing 0 j 

Educators find that what they thought in the past was adequate j 



reporting on successes Cand failures occasionally) has proved 
to be too little^ too late in many instances; and despite com- 
mitment at the highest levels 9 vocational education has been 
the subject of continuous heated and at times even bitter debate** 
This debate centers on issues such as usefulness or adequacy of 
vocational education 0 Do graduates really use the training they 
receive in school? How effective is this training? Does it 
really prepare for the types of jobs young people obtain on 
graduation? Should training be conducted in comprehensive or 
separate vocational high schools? Another issue often raised 
concerns the image of vocational education which is seen by some 
observers as appropriate only for those students who cannot 
succeed in the more demanding academic curriculum. Another 
point which is being emphasized tiday is the role of vocational 
education as it relates to minority groups and ethers from dis- 
advantaged backgrounds. We need to regard the elevation of 
entry skill levels as a focal point for some of our less able 
students. 
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CHAPTER II. 



EVALUATION TECHNIQUES 

How does the average teacher of any vocational-technical 
class evaluate his students? Generally, it is done with teacher** 
made tests which have dubious value except to regurgitate those 
phases of a program that have been emphasized enough to establish 
a basis for recall. True, there are some standardized tests in 
academic subjects but rarely are these adapted to vocational- 
technical education. Tests in math are a good example. 

How does the -average instructor evaluate his program in 
total? The current concept is that he either doesn’t or he 
bases his evaluation on test scores of his successful students, 
or placement (not retention) of jobs his students have. The 
literature points an accusing finger at vocational education 
for lack of evaluation of the kind that is descriptive and 
definitive of what is actually happening in the vocational- 
technical program. Jerome - Moss, Jr. places the onus squarely 
on the shoulders of vocational-technical educators. 

The following ore some evaluation techniques that have 

been described in recent research: 

X. Graduate interviews : This is a process whereby all 

graduates of a school are interviewed as to job plans, school 
plans, military or marriage plans before they leave school. 

This is a tructured interview technique so the data can be 
catalogued and related to programs. The point of interest here 
is the ratio of job to school, etc., plans of the vocational 
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student as opposed to the .general or academic gradua«*e<> It 
Has been contended in the past that the vocational graduage Has 
a much better job concept ■formulated than the general or aca- 
demic graduate. This interview process is counselor oriented 
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and very costly. 

2„ Career follow-up study ; One of the most commonly 
used techniques ® career follow— up study probably produces the 
greatest quantity of data today. Some of the relevance of the 
data is under heavy fire® but this type of study seemed to be 
the easiest to use to establish (erroneously or not) that voca- 
tional-technical programs were in fact producing a useable pro- 
duct (students placed in jobsK 

Seme of the items evaluated were: 

a, Relatedness of first job to trade student 
b 0 Assessment of skill origins and requisites 
Co Past high school® occupational history 
do Off-the-job interests and affiliations 
Bo Leisure activities 
fo Organization affiliations 

Sharp and Krasnegor* comment concerning follow studies 
of the more common descriptive variety® 'Loostudies can be good 
or badp valid or invalid and yield new insights or document the 

obvious* tt 

Sharp further commented that the more complex studies of 
follow-up were offering hopeful signs that the narrow and self- 
conscious "placement” criteria were being replaced with more 
sophisticated evaluation devices which take a wider view of 
employment outcomes and attempt to relate specific training 
approaches to identifiable changes m the trainee 9 s work lifeo 





Eninger stated in a recent study related to fallow-up 

techniques, ?, If the graduated first job is not in the trade 

studied or highly related, the chances nre high that he will 
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never enter the trade or a highly related trade®" It can be 
suggested that at best placement data from follqw-up gives the 
school administrator speedy and inexpensive feedback highlight- 
ing the extremes of program success and failure® An aspect of 
this relates to the disproportionate number of students re= 
cruited for vocational education from disadvantaged or low- 
achieving groups and then trying to establish a statistical 
inference related to general education dropouts or job success 
of other kinds of students® 

g 

3® Achievement testing ; Harry Davis states that the 
process of learning in vocational-technical education is no 
different from any other instructional area® He assumes p there- 
fore . to test the behavioral changes that occur with stan- 
dardized instruments designed by the Ohio Trade and Industrial 
Supervisors Workshop of 1950., These tests have been used in 
eight states® There are seven basic achievement tests in the 
series, each related to a different trade field® The develop- 
ment methods do not appear as careful and sophisticated as the 
major test developers might use, but the tests could be con- 
sidered as a useful taolo 

4® State and national licensing examinations s Here is 
an interesting technique that works from the ultimate product 
at appoint of no return ® " The product of the program (student) 
is evaluated for licensure after he has passed all the evaluative 
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devices that the program imposes t much like a doctor or dentzst 
would be licensed but as a sub—prof essianal levelo The value 
of’ such procedure is questionable as two years plus of education 
in many respects are being evaluated as a total in one examina- 
tion,, It could be questioned what such exoamination really 

10 

evaluates at the point at which it is administered,, 

5° Industry advisory committees s To the true believer 
in vocational educations the use of the advisory committee as 
part of a program is a “musts ” To use it as an assisting device 
in evaluation makes good sense,. The question is raised as to 
whether or not a local advisory committee would '’tend to white- 
wash” a program it was vitally concerned with B and Mr* 3urt 
comments that it has been his experience to find the reverse,, 
They tend to be vitally interested in having the best possible 
program as a result of their efforts? They will in effect 
criticize its weakness and recommend change,, 

6* Ik® systems, approach cycles The computer has changed 
concepts of evaluation to the extent that models are being de- 
veloped on a systems analysis approach,. This allows programs 
to be broken down into digestable segments which can be dealt 
with -cn a system flow basiso As the programs are disassembled 
into flow unitsp objectives have to be analyzed into units that 
permit study* Two major areas of problem definition and problem 
solving face the evaluator as he establishes a systems approach,, 
7* Accreditation as an evaluation techniqMg. * Accredi- 
tation of a school involves the total concept of the school m 
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ssli aspectso It was the practice for a time to ignore the 

I. 

vocational-technical aspects of a schooi , s program as not being 
a part of the process of accreditation because accx'sditation per 
se was related to an acceptance function by colleges and univer- 
sitieso This acceptance function has given way to total evalua- 
tion from the standpoint of fulfilling the student 9 s needs*, the 

community 5 s participation v etc. All nationally recognized 

1 3 

accrediting agencies now follow four functions s 

1 o Th^ publicly establish the standards by which they 
will judge quality® 

2 . They send qualified educators to serve as inspectors 
on visits. 

3© They approve and include on their lists of accredited 
institutions only those which satisfactorily meet the 
standards. 

4® They revisit and revaluate these institutions per- 
iodically and remove from their lists any institu- 
tion that is not continuing to meet their standards© 

From these statements of intent the agencies that do 
accreditation state that now they do not accredit vocational- 
technical education separately but as a part of the total insti- 
tution© 

F rom these points of view has come an expression that 

"The technical institute that turns out excellent technicians 

is a quality institution just as the graduate school producing 

fine scholars at a doctoral level are quality institutions© 

©.©Each institution must be evaluated in terms of the function 

1 3 

it proposes to perform© ” 

The Southern Association . of Colleges and Schools in a 
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publication of May, 19.67, 14 said in part, "...Vocational schools 
in particular are in the mainstream of change and should be sen- 
sitive to it. Some [educators] fear that accreditation would 
lessen this sensitivity, bring rigidity and create conflict be- 
tween the needs of business and industry on the one hand, and the 
standards of the accrediting association on the other," 

3- Self initiated evaluatio n! This technique has, built 
in, a number of elements that relate to local program study: 
a. Selection of competent leadership 

Involvement of faculty in the evaluation process 

"V.. 

A focus on ^he output of programs 

do Identification of real objectives of the 
total program 

e. Use appropriate methods of data gathering 

f. Study the essential elements of programs 
g* Involving citizen groups 

Several other major points Byram 15 emphasizes are development 
of faculty understanding of the evaluation because of the threaten- 
ed feeling many faculties have whenever evaluation is mentioned. 
Some of this threatened feeling comes from unhappy previous exper- 
iences. They need to be assured that program evaluation is not 
teacher evaluation. Michigan State has developed an instrument 
called the "Vocational Education Understandings Inventory," which 
was designed to alleviate areas of misunderstanding and threat to 
teachers. Another point is the care that should be used in deter- 
mining expected outcomes based purely or. societal or local needs. 

Eninger states, "Schools are for people and are not to be thought 

„6 

of as tools of the labor force or the economy. 
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9* Personality change as a result of, o c c u pa,tjj3jial. 

16 

education and evaluation s Joseph Champagne states , ’’The position 
I adhere to is that occupational education must give focus to 
educating the entire man *•« so that the individual appreciates 
the dignity of his worth* <>**The individual needs to be educated 
to function with a newly acquired skill rather than to be merely 
trained in a skillo” He further states, ’’There is a noticeable 
lack of discussion of techniques of attitude assessment speci- 
fically in occupational education 0 ” He alludes to the extensive 
efforts related to adult education orientation to develop sound 
attitudes toward reception of training, but the general ignoring 
of this same phase of pre-training in our high schools o He 
suggests this process could be conceptualized as achievement 
motivation which needs to be aroused and nurtured with some self- 
perpetuating force that carries the student through as much of 
his work life as pcssible 0 

1 * 

IGo Eval uative crit eria; Ralph Tyler in a recent article 

* **’ Ty. ~ y r T HT1 I f~ I ' I ~T W ■|*L.X I . ■ J J ** t» * I— -l>Q *Wd3 * 

listed three kinds of evaluation criteria that may be used to 

judge schools and programs 0 

ai Structure - Facilities, plant and equipment 

b c Process - The way in which the school carries on 

c. Product - Ultimate evaluation — how far does the 

school carry out what it tries to 
accomplish 

The National Study of Secondary School Evaluation suggests 
that areas of program and schools be evaluated in terms of its 
objectives and related to how the reaching of those objectives 
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fills the needs of its students*. 

Many other criteria could be formulated that could be 
related to the specifics of course content and objectives as 
well as to an entire program or school philosophy. 
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CHAPTER III 



CONCLUSIONS 

It becomes obvious from the literature that the problem 
of evaluating vocational-technical education to determine its 
effectiveness , content, and future is moving forward with ever 
increasing speedo Conferences such as the Upjon Conference in 
Atlantic City on October 6-9, 1968, are but indicators on the 
horizon of what is to coiubo Evaluation is to be the keystone 
of future programs and it is felt and could safely be predicted 
that future programs will of necessity have built-in evaluation 
programs and techniques that will provide a continuing fund of 
data as the program evolves 0 

The legislatures, boards and the congress are no longer 
content to wait ten years or more before educators decide to 
find out if the programs are in fact doing what was claimed 
for them at the outset. There are warning signs along the way 
that education must need or it will find outside sources will 
be brought in to do what education has failed to do for itself® 

There are some pluses too for the educator and programs® 

As it can be established that programs are or are not function®* 
ing as a result of immediate evaluation, adjustments can be made 
which will make the utilization of facilities and funds as 
economical as possible® This will provide for a dynamic attitude 
development within education as educators recognize the merits 
of real flexibility within their institutions. 
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